Route analytics queries by index setting, not table-name prefix by bowenlan-amzn · Pull Request #5429 · opensearch-project/sql

bowenlan-amzn · 2026-05-10T04:30:13Z

Description

Today `RestUnifiedQueryAction.isAnalyticsIndex` routes queries to the analytics engine when the source index name starts with `parquet_`. That's brittle — it conflates naming convention with storage type: an index created without the prefix but with pluggable dataformat enabled is silently sent to the Lucene path, and an index named `parquet_foo` without the setting is mis-dispatched.

Switch to the authoritative signal: the `index.pluggable.dataformat.enabled` cluster-state setting. This is the same flag integration tests (`CoordinatorReduceIT`, `CompositeCommitDeletionIT`) use to create analytics-backed indices, and what `FieldStorageResolver` reads to resolve field storage.

Routing behavior

`index.pluggable.dataformat.enabled`	Dispatch
`true`	analytics engine (DataFusion via `QueryPlanExecutor`)
absent / `false` / index missing	sql-plugin Calcite → OpenSearch DSL path

Issues Resolved

Mustang rollout pre-work — aligns PPL/SQL routing with the index-setting-based model already used elsewhere in the engine.

Check List

New functionality includes testing.
New functionality has been documented. (javadoc on `isAnalyticsIndex`)
Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2026-05-10T04:31:10Z

PR Reviewer Guide 🔍

(Review updated until commit `f2e83ee`)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review Possible Issue The method `isPluggableDataformatIndex` returns false when `indexMetadata` is null (index does not exist), but the calling code in `isAnalyticsIndex` catches all exceptions and returns false. If `indexMetadata.getSettings()` or the setting getters throw an exception (e.g., due to malformed settings), the exception is silently swallowed. This could mask configuration errors or unexpected states, making debugging difficult when an index should route to analytics but doesn't due to a settings issue. private boolean isPluggableDataformatIndex(String indexName) { var indexMetadata = clusterService.state().metadata().index(indexName); if (indexMetadata == null) { return false; } var settings = indexMetadata.getSettings(); return IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(settings) && "parquet".equals(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.get(settings)); }

github-actions · 2026-05-10T04:31:23Z

PR Code Suggestions ✨

Latest suggestions up to f2e83ee

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Cache cluster state reference The method accesses cluster state without synchronization, which could lead to race conditions if the cluster state changes between the null check and settings retrieval. Consider caching the cluster state reference or using a snapshot to ensure consistency throughout the method execution. plugin/src/main/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryAction.java [106-114] private boolean isPluggableDataformatIndex(String indexName) { - var indexMetadata = clusterService.state().metadata().index(indexName); + var clusterState = clusterService.state(); + var indexMetadata = clusterState.metadata().index(indexName); if (indexMetadata == null) { return false; } var settings = indexMetadata.getSettings(); return IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(settings) && "parquet".equals(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.get(settings)); } Suggestion importance[1-10]: 4 __ Why: While caching the cluster state reference is a minor improvement for code clarity, the concern about race conditions is overstated. The `clusterService.state()` already returns a consistent snapshot, and the operations are read-only. The suggestion provides marginal benefit in terms of readability but doesn't address a critical issue.	Low

Previous suggestions

Suggestions up to commit 7f221e1

Category	Suggestion	Impact
Possible issue	Add null-safety for cluster state Add null-safety check for `clusterService.state()` and `metadata()` to prevent potential `NullPointerException` when cluster state is unavailable. This ensures the method gracefully handles cluster state initialization or unavailability scenarios. plugin/src/main/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryAction.java [105-109] private boolean isPluggableDataformatIndex(String indexName) { - var indexMetadata = clusterService.state().metadata().index(indexName); + var clusterState = clusterService.state(); + if (clusterState == null \|\| clusterState.metadata() == null) { + return false; + } + var indexMetadata = clusterState.metadata().index(indexName); return indexMetadata != null && IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(indexMetadata.getSettings()); } Suggestion importance[1-10]: 6 __ Why: The suggestion adds defensive null checks for `clusterService.state()` and `metadata()`, which could prevent potential `NullPointerException` in edge cases. However, in typical OpenSearch plugin contexts, these are rarely null during normal operation, making this a minor defensive improvement rather than a critical fix.	Low

Suggestions up to commit f2bc431

Category	Suggestion	Impact
Possible issue	Add null-safety for cluster state Add null-safety check for `clusterService.state()` and `metadata()` to prevent potential `NullPointerException`. The cluster state or metadata could be null during cluster initialization or shutdown scenarios. plugin/src/main/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryAction.java [108-114] private boolean isPluggableDataformatIndex(String indexName) { - var indexMetadata = clusterService.state().metadata().index(indexName); + var clusterState = clusterService.state(); + if (clusterState == null \|\| clusterState.metadata() == null) { + return false; + } + var indexMetadata = clusterState.metadata().index(indexName); if (indexMetadata == null) { return false; } return indexMetadata.getSettings().getAsBoolean(PLUGGABLE_DATAFORMAT_ENABLED_SETTING, false); } Suggestion importance[1-10]: 5 __ Why: While adding null checks for `clusterService.state()` and `metadata()` could prevent potential NPEs in edge cases, this is a defensive programming suggestion rather than a critical bug fix. The `ClusterService` typically guarantees non-null state in normal operation, and the existing null check for `indexMetadata` already handles the most common failure case.	Low

Suggestions up to commit cbc0ca7

Category	Suggestion	Impact
Possible issue	Add exception handling for cluster state access The method accesses cluster state without proper synchronization or error handling. Cluster state operations can throw exceptions if the cluster is in an unstable state. Wrap the cluster state access in a try-catch block to prevent unexpected failures from propagating. plugin/src/main/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryAction.java [105-111] private boolean isPluggableDataformatIndex(String indexName) { - var indexMetadata = clusterService.state().metadata().index(indexName); - if (indexMetadata == null) { + try { + var indexMetadata = clusterService.state().metadata().index(indexName); + if (indexMetadata == null) { + return false; + } + return indexMetadata.getSettings().getAsBoolean(PLUGGABLE_DATAFORMAT_ENABLED_SETTING, false); + } catch (Exception e) { return false; } - return indexMetadata.getSettings().getAsBoolean(PLUGGABLE_DATAFORMAT_ENABLED_SETTING, false); } Suggestion importance[1-10]: 5 __ Why: While adding error handling for cluster state access is a reasonable defensive practice, the suggestion doesn't provide strong evidence that `clusterService.state().metadata().index()` commonly throws exceptions in this context. The score reflects that this is a minor improvement for robustness rather than fixing a critical bug.	Low

github-actions · 2026-05-10T04:49:19Z

Persistent review updated to latest commit f2bc431

Today `RestUnifiedQueryAction.isAnalyticsIndex` dispatches to the analytics engine when the source index name starts with `parquet_`. That's brittle — it conflates naming convention with storage type. An index created without the prefix but with pluggable dataformat enabled is silently sent to the Lucene path; an index named `parquet_foo` without the setting is mis-dispatched to analytics. Use the authoritative signal instead: the `index.pluggable.dataformat.enabled` setting on cluster-state metadata. This is the same setting integration tests (`CoordinatorReduceIT`, `CompositeCommitDeletionIT`, etc.) already use to create analytics-backed indices, and it's what `FieldStorageResolver` reads to decide field-level storage. Behavior: - `index.pluggable.dataformat.enabled=true` → analytics engine (DataFusion) - flag absent / false / index missing → Calcite→OpenSearch DSL path Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

github-actions · 2026-05-10T05:57:47Z

Persistent review updated to latest commit 7f221e1

expani · 2026-05-10T19:57:51Z

+  private boolean isPluggableDataformatIndex(String indexName) {
+    var indexMetadata = clusterService.state().metadata().index(indexName);
+    return indexMetadata != null
+        && IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(indexMetadata.getSettings());


"index.pluggable.dataformat.enabled": true, "index.pluggable.dataformat": "composite", "index.composite.primary_data_format": "parquet"

I think we also need to check on index.composite.primary_data_format which would indicate if this is Parquet backed index.

If the pluggable dataformat is enabled index.pluggable.dataformat.enabled , but index.composite.primary_data_format : lucene , then we can still go the old route until AnalyticsPlugin supports pure Lucene Indices and DocValues.

Only route to the analytics path when both pluggable.dataformat.enabled=true AND pluggable.dataformat=parquet. If the format is lucene (or anything else), fall through to the standard Calcite→DSL path. Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

github-actions · 2026-05-10T20:30:32Z

Persistent review updated to latest commit f2e83ee

bowenlan-amzn · 2026-05-11T16:51:38Z

+    }
+    var settings = indexMetadata.getSettings();
+    return IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(settings)
+        && "parquet".equals(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.get(settings));


PLUGGABLE_DATAFORMAT_VALUE_SETTING should be composite

@ignore

Single squashed delivery of the long-running feature/mustang-ppl-integration branch into main, consolidating 22 feature-branch PRs plus the conflict-resolved merge of current main. Squashed because the feature branch's history includes commits with missing or mismatched Signed-off-by trailers that block DCO at this scope — the equivalent issue documented for the catch-up squashes (opensearch-project#5397). The feature branch f006e29 is retained for individual-commit lineage. ### What this delivers Analytics-engine PPL integration — a new execution path that routes Parquet-backed (non-Lucene) indices through an analytics engine while keeping Lucene-backed indices on the existing v2 / Calcite paths. Headline pieces: - Query routing (opensearch-project#5267) — PPL queries against Parquet-backed indices hand off to the analytics-engine execution path; Lucene-backed indices continue through the legacy path - Explain support (opensearch-project#5275) — EXPLAIN covers the analytics-engine path - Profiling + UnifiedQueryParser (opensearch-project#5285) — migrates PPL parsing to the unified parser and wires profiling metrics through the analytics path - extendedPlugins wiring (opensearch-project#5302) — analytics-engine attaches as an OpenSearch extension via SPI - SQL REST endpoint integration (opensearch-project#5317) — same analytics-route fork applied to the SQL transport, plus delegateToV2Engine extraction in RestSqlAction - Async QueryPlanExecutor (opensearch-project#5396) — async execution for analytics-engine plans + version bump to OpenSearch 3.7 - Optional dependency (opensearch-project#5403) — analytics-engine becomes an optional runtime dep so the SQL bundle is shippable without it - Index-setting-based routing (opensearch-project#5429) — replaces the earlier table-name-prefix heuristic with an authoritative index-setting check Supporting infrastructure: - Gradle wrapper bump to 9.4.1 (opensearch-project#5406) - Jar-hell exclusions for arrow-flight-rpc / httpcore5-h2 / httpcore5-reactive / httpclient5 (opensearch-project#5400, opensearch-project#5409) - IT plumbing: CalciteEvalCommandIT / CalciteFieldFormatCommandIT carried through the helper-managed index path (opensearch-project#5407, opensearch-project#5417); CalciteReplaceCommandIT column-order-agnostic (opensearch-project#5415); @ignore'd Calcite ITs dropped from CalciteNoPushdownIT (opensearch-project#5416) - plugins.calcite.enabled=true defaulted on the unified query path (opensearch-project#5413) - PPL_REX_MAX_MATCH_LIMIT bridged into UnifiedQueryContext (opensearch-project#5418) - Calcite tolerance fixes: array() default type (opensearch-project#5421), containsNestedAggregator flat-leaf schemas (opensearch-project#5423) - Sandbox deps switched to analytics-api JDK 21 surface (opensearch-project#5426) ### Feature-branch commits squashed (22) opensearch-project#5429, opensearch-project#5426, opensearch-project#5423, opensearch-project#5421, opensearch-project#5418, opensearch-project#5403, opensearch-project#5417, opensearch-project#5415, opensearch-project#5416, opensearch-project#5413, opensearch-project#5407, opensearch-project#5409, opensearch-project#5406, opensearch-project#5400, opensearch-project#5396, opensearch-project#5317, opensearch-project#5302, opensearch-project#5285, opensearch-project#5275, opensearch-project#5267, opensearch-project#5397, opensearch-project#5286 ### Main commits absorbed via the merge (54) Brings the branch up to current upstream/main (54 commits since the last catch-up at opensearch-project#5397, divergence point 513e1b2). Highlights: opensearch-project#5419, opensearch-project#5408, opensearch-project#5414, opensearch-project#5399, opensearch-project#5394, opensearch-project#5361, opensearch-project#5360, opensearch-project#5240, opensearch-project#5266, opensearch-project#5278, plus 44 others (bugfixes, doc updates, infra). ### Conflict resolutions (7) Resolved during the merge of main into the feature branch. Resolution kept the feature branch's analytics-engine-path semantics where main's changes would have regressed them. - api/.../UnifiedQueryContext.java Blank-line-only conflict; took main's tighter formatting. - core/.../executor/QueryService.java Kept feature's CalciteClassLoaderHelper.withCalciteClassLoader(...) wrapping (required for analytics-engine classloader isolation) and the matching import. - integ-test/build.gradle Kept feature's detailed root-cause comment on the Gradle 9.4.1 TestEventReporterAsListener workaround; kept ASCII ordering of JSONRequestIT / JoinIT and SQLFunctionsIT / ShowIT / SourceFieldIT entries. - integ-test/.../CalciteEvalCommandIT.java Kept feature's if (!TestUtils.isIndexExist(...)) idempotency guards on test_eval and test_eval_agent setup (needed for the helper-managed index analytics-engine compatibility run). - legacy/.../RestSqlAction.java Kept feature's delegateToV2Engine(...) (extracted from the analytics-engine routing path). Both sides added handleException / getRestStatus / getRawErrorCode; removed the duplicate set git produced. - plugin/.../SQLPlugin.java Took the union of imports: ExecutionEngine + ExecutionEngine.ExplainResponse + QueryType. - plugin/.../transport/TransportPPLQueryAction.java Combined main's OpenSearchPluginModule(extensionsHolder.engines()) and feature's local pluginSettings / pluginSettingsRef wiring. EngineExtensionsHolder.java is a new file from main (opensearch-project#5298) preserved as-is. ### Compatibility / opt-in The analytics-engine path is gated by the extendedPlugins extension being installed (opensearch-project#5403 makes the dep optional). Clusters without analytics-engine installed see no behavior change. Clusters with analytics-engine installed route only Parquet-backed indices through the new path (opensearch-project#5429 — by index setting). ### Verification - ./gradlew :api:compileJava :core:compileJava :legacy:compileJava :opensearch-sql-plugin:compileJava :integ-test:compileTestJava passes locally Signed-off-by: Kai Huang <ahkcs@amazon.com>

@ignore

Single squashed delivery of the long-running feature/mustang-ppl-integration branch into main, consolidating 22 feature-branch PRs plus the conflict-resolved merge of current main. Squashed because the feature branch's history includes commits with missing or mismatched Signed-off-by trailers that block DCO at this scope — the equivalent issue documented for the catch-up squashes (opensearch-project#5397). The feature branch f006e29 is retained for individual-commit lineage. ### What this delivers Analytics-engine PPL integration — a new execution path that routes Parquet-backed (non-Lucene) indices through an analytics engine while keeping Lucene-backed indices on the existing v2 / Calcite paths. Headline pieces: - Query routing (opensearch-project#5267) — PPL queries against Parquet-backed indices hand off to the analytics-engine execution path; Lucene-backed indices continue through the legacy path - Explain support (opensearch-project#5275) — EXPLAIN covers the analytics-engine path - Profiling + UnifiedQueryParser (opensearch-project#5285) — migrates PPL parsing to the unified parser and wires profiling metrics through the analytics path - extendedPlugins wiring (opensearch-project#5302) — analytics-engine attaches as an OpenSearch extension via SPI - SQL REST endpoint integration (opensearch-project#5317) — same analytics-route fork applied to the SQL transport, plus delegateToV2Engine extraction in RestSqlAction - Async QueryPlanExecutor (opensearch-project#5396) — async execution for analytics-engine plans + version bump to OpenSearch 3.7 - Optional dependency (opensearch-project#5403) — analytics-engine becomes an optional runtime dep so the SQL bundle is shippable without it - Index-setting-based routing (opensearch-project#5429, opensearch-project#5432) — replaces the earlier table-name-prefix heuristic with an authoritative index-setting check Supporting infrastructure: - Gradle wrapper bump to 9.4.1 (opensearch-project#5406) - Jar-hell exclusions for arrow-flight-rpc / httpcore5-h2 / httpcore5-reactive / httpclient5 (opensearch-project#5400, opensearch-project#5409) - IT plumbing: CalciteEvalCommandIT / CalciteFieldFormatCommandIT carried through the helper-managed index path (opensearch-project#5407, opensearch-project#5417); CalciteReplaceCommandIT column-order-agnostic (opensearch-project#5415); @ignore'd Calcite ITs dropped from CalciteNoPushdownIT (opensearch-project#5416) - plugins.calcite.enabled=true defaulted on the unified query path (opensearch-project#5413) - PPL_REX_MAX_MATCH_LIMIT bridged into UnifiedQueryContext (opensearch-project#5418) - Calcite tolerance fixes: array() default type (opensearch-project#5421), containsNestedAggregator flat-leaf schemas (opensearch-project#5423) - Sandbox deps switched to analytics-api JDK 21 surface (opensearch-project#5426) ### Feature-branch commits squashed (22) opensearch-project#5432, opensearch-project#5429, opensearch-project#5426, opensearch-project#5423, opensearch-project#5421, opensearch-project#5418, opensearch-project#5403, opensearch-project#5417, opensearch-project#5415, opensearch-project#5416, opensearch-project#5413, opensearch-project#5407, opensearch-project#5409, opensearch-project#5406, opensearch-project#5400, opensearch-project#5396, opensearch-project#5317, opensearch-project#5302, opensearch-project#5285, opensearch-project#5275, opensearch-project#5267, opensearch-project#5397, opensearch-project#5286 ### Main commits absorbed via the merge (54) Brings the branch up to current upstream/main (54 commits since the last catch-up at opensearch-project#5397, divergence point 513e1b2). Highlights: opensearch-project#5419, opensearch-project#5408, opensearch-project#5414, opensearch-project#5399, opensearch-project#5394, opensearch-project#5361, opensearch-project#5360, opensearch-project#5240, opensearch-project#5266, opensearch-project#5278, plus 44 others (bugfixes, doc updates, infra). ### Conflict resolutions (7) Resolved during the merge of main into the feature branch. Resolution kept the feature branch's analytics-engine-path semantics where main's changes would have regressed them. - api/.../UnifiedQueryContext.java Blank-line-only conflict; took main's tighter formatting. - core/.../executor/QueryService.java Kept feature's CalciteClassLoaderHelper.withCalciteClassLoader(...) wrapping (required for analytics-engine classloader isolation) and the matching import. - integ-test/build.gradle Kept feature's detailed root-cause comment on the Gradle 9.4.1 TestEventReporterAsListener workaround; kept ASCII ordering of JSONRequestIT / JoinIT and SQLFunctionsIT / ShowIT / SourceFieldIT entries. - integ-test/.../CalciteEvalCommandIT.java Kept feature's if (!TestUtils.isIndexExist(...)) idempotency guards on test_eval and test_eval_agent setup (needed for the helper-managed index analytics-engine compatibility run). - legacy/.../RestSqlAction.java Kept feature's delegateToV2Engine(...) (extracted from the analytics-engine routing path). Both sides added handleException / getRestStatus / getRawErrorCode; removed the duplicate set git produced. - plugin/.../SQLPlugin.java Took the union of imports: ExecutionEngine + ExecutionEngine.ExplainResponse + QueryType. - plugin/.../transport/TransportPPLQueryAction.java Combined main's OpenSearchPluginModule(extensionsHolder.engines()) and feature's local pluginSettings / pluginSettingsRef wiring. EngineExtensionsHolder.java is a new file from main (opensearch-project#5298) preserved as-is. ### Compatibility / opt-in The analytics-engine path is gated by the extendedPlugins extension being installed (opensearch-project#5403 makes the dep optional). Clusters without analytics-engine installed see no behavior change. Clusters with analytics-engine installed route only Parquet-backed indices through the new path (opensearch-project#5429 — by index setting). ### Verification - ./gradlew :api:compileJava :core:compileJava :legacy:compileJava :opensearch-sql-plugin:compileJava :integ-test:compileTestJava passes locally Signed-off-by: Kai Huang <ahkcs@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>

@ignore

* Land analytics-engine PPL integration into main Single squashed delivery of the long-running feature/mustang-ppl-integration branch into main, consolidating 22 feature-branch PRs plus the conflict-resolved merge of current main. Squashed because the feature branch's history includes commits with missing or mismatched Signed-off-by trailers that block DCO at this scope — the equivalent issue documented for the catch-up squashes (#5397). The feature branch f006e29 is retained for individual-commit lineage. ### What this delivers Analytics-engine PPL integration — a new execution path that routes Parquet-backed (non-Lucene) indices through an analytics engine while keeping Lucene-backed indices on the existing v2 / Calcite paths. Headline pieces: - Query routing (#5267) — PPL queries against Parquet-backed indices hand off to the analytics-engine execution path; Lucene-backed indices continue through the legacy path - Explain support (#5275) — EXPLAIN covers the analytics-engine path - Profiling + UnifiedQueryParser (#5285) — migrates PPL parsing to the unified parser and wires profiling metrics through the analytics path - extendedPlugins wiring (#5302) — analytics-engine attaches as an OpenSearch extension via SPI - SQL REST endpoint integration (#5317) — same analytics-route fork applied to the SQL transport, plus delegateToV2Engine extraction in RestSqlAction - Async QueryPlanExecutor (#5396) — async execution for analytics-engine plans + version bump to OpenSearch 3.7 - Optional dependency (#5403) — analytics-engine becomes an optional runtime dep so the SQL bundle is shippable without it - Index-setting-based routing (#5429, #5432) — replaces the earlier table-name-prefix heuristic with an authoritative index-setting check Supporting infrastructure: - Gradle wrapper bump to 9.4.1 (#5406) - Jar-hell exclusions for arrow-flight-rpc / httpcore5-h2 / httpcore5-reactive / httpclient5 (#5400, #5409) - IT plumbing: CalciteEvalCommandIT / CalciteFieldFormatCommandIT carried through the helper-managed index path (#5407, #5417); CalciteReplaceCommandIT column-order-agnostic (#5415); @ignore'd Calcite ITs dropped from CalciteNoPushdownIT (#5416) - plugins.calcite.enabled=true defaulted on the unified query path (#5413) - PPL_REX_MAX_MATCH_LIMIT bridged into UnifiedQueryContext (#5418) - Calcite tolerance fixes: array() default type (#5421), containsNestedAggregator flat-leaf schemas (#5423) - Sandbox deps switched to analytics-api JDK 21 surface (#5426) ### Feature-branch commits squashed (22) #5432, #5429, #5426, #5423, #5421, #5418, #5403, #5417, #5415, #5416, #5413, #5407, #5409, #5406, #5400, #5396, #5317, #5302, #5285, #5275, #5267, #5397, #5286 ### Main commits absorbed via the merge (54) Brings the branch up to current upstream/main (54 commits since the last catch-up at #5397, divergence point 513e1b2). Highlights: #5419, #5408, #5414, #5399, #5394, #5361, #5360, #5240, #5266, #5278, plus 44 others (bugfixes, doc updates, infra). ### Conflict resolutions (7) Resolved during the merge of main into the feature branch. Resolution kept the feature branch's analytics-engine-path semantics where main's changes would have regressed them. - api/.../UnifiedQueryContext.java Blank-line-only conflict; took main's tighter formatting. - core/.../executor/QueryService.java Kept feature's CalciteClassLoaderHelper.withCalciteClassLoader(...) wrapping (required for analytics-engine classloader isolation) and the matching import. - integ-test/build.gradle Kept feature's detailed root-cause comment on the Gradle 9.4.1 TestEventReporterAsListener workaround; kept ASCII ordering of JSONRequestIT / JoinIT and SQLFunctionsIT / ShowIT / SourceFieldIT entries. - integ-test/.../CalciteEvalCommandIT.java Kept feature's if (!TestUtils.isIndexExist(...)) idempotency guards on test_eval and test_eval_agent setup (needed for the helper-managed index analytics-engine compatibility run). - legacy/.../RestSqlAction.java Kept feature's delegateToV2Engine(...) (extracted from the analytics-engine routing path). Both sides added handleException / getRestStatus / getRawErrorCode; removed the duplicate set git produced. - plugin/.../SQLPlugin.java Took the union of imports: ExecutionEngine + ExecutionEngine.ExplainResponse + QueryType. - plugin/.../transport/TransportPPLQueryAction.java Combined main's OpenSearchPluginModule(extensionsHolder.engines()) and feature's local pluginSettings / pluginSettingsRef wiring. EngineExtensionsHolder.java is a new file from main (#5298) preserved as-is. ### Compatibility / opt-in The analytics-engine path is gated by the extendedPlugins extension being installed (#5403 makes the dep optional). Clusters without analytics-engine installed see no behavior change. Clusters with analytics-engine installed route only Parquet-backed indices through the new path (#5429 — by index setting). ### Verification - ./gradlew :api:compileJava :core:compileJava :legacy:compileJava :opensearch-sql-plugin:compileJava :integ-test:compileTestJava passes locally Signed-off-by: Kai Huang <ahkcs@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> * Address @penghuo: revert stray blank line in doctest/build.gradle After 'apply plugin: opensearch.testclusters', one blank line is enough — restoring the single-blank spacing to match upstream/main. Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>

bowenlan-amzn force-pushed the experiment/routing-by-dataformat-setting branch from cbc0ca7 to f2bc431 Compare May 10, 2026 04:48

bowenlan-amzn force-pushed the experiment/routing-by-dataformat-setting branch from f2bc431 to 7f221e1 Compare May 10, 2026 05:56

bowenlan-amzn marked this pull request as ready for review May 10, 2026 06:06

bowenlan-amzn requested review from LantaoJin, RyanL1997, Swiddis, acarbonetto, ahkcs, anirudha, dai-chen, joshuali925, mengweieric, noCharger, penghuo, ps48, qianheng-aws, songkant-aws, vamsimanohar, ykmr1224 and yuancu as code owners May 10, 2026 06:06

expani reviewed May 10, 2026

View reviewed changes

expani approved these changes May 10, 2026

View reviewed changes

ahkcs merged commit f006e29 into opensearch-project:feature/mustang-ppl-integration May 11, 2026
34 of 36 checks passed

bowenlan-amzn commented May 11, 2026

View reviewed changes

This was referenced May 11, 2026

Land analytics-engine PPL integration into main #5431

Closed

Land analytics-engine PPL integration into main #5430

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Route analytics queries by index setting, not table-name prefix#5429

Route analytics queries by index setting, not table-name prefix#5429
ahkcs merged 2 commits into
opensearch-project:feature/mustang-ppl-integrationfrom
bowenlan-amzn:experiment/routing-by-dataformat-setting

bowenlan-amzn commented May 10, 2026

Uh oh!

github-actions Bot commented May 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

expani May 10, 2026

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

Uh oh!

bowenlan-amzn May 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bowenlan-amzn commented May 10, 2026

Description

Issues Resolved

Check List

Uh oh!

github-actions Bot commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit f2e83ee)

Uh oh!

github-actions Bot commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Previous suggestions

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

expani May 10, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

Uh oh!

bowenlan-amzn May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented May 10, 2026 •

edited

Loading

(Review updated until commit `f2e83ee`)

github-actions Bot commented May 10, 2026 •

edited

Loading